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1  Objectives 

The  objectives  of  AFOSR  Grant  No.  F49620-03-1-0129  were  as  follows: 

1 .  Develop,  implement,  and  enhance  probabilistic  model-building  GAs  for  non-binary 
codes. 

2.  Extend  existing  facetwise  models  to  non-binary  codes. 

3.  Extend  bounding  test  functions  to  non-binary  codes. 

4.  Extend  the  proposed  non-binary  algorithms  to  hierarchically  difficult  problems. 

5.  Apply  the  developed  algorithms  to  two  problems  of  Air  Force  interest. 

These  goals  have  been  substantially  accomplished.  Moreover,  new  findings  led  to  important 
discoveries  that  were  unanticipated  at  the  beginning  of  the  project.  The  report  starts  with  a 
retrospective  examination  of  project  accomplishment. 

2  Retrospective  Examination  of  Accomplishment 

Efficiency  enhancement  from  integrated  model  building.  A  fundamental  surprise  was  the 
discovery  that  integrating  model  building  with  various  forms  of  efficiency  enhancement  results 
in  orders  of  magnitude  of  speedup  over  that  expected  using  naive  implementations.  In  particular 
endogenous  fitness  modeling  and  an  adaptive  continuation  operator  showed  remarkable  speedups 
(1-2  orders  of  magnitude)  and  a  renewal  proposal  was  prepared  and  has  been  funded  to  explore 
the  extensions  of  these  important  findings. 

liBOA  solves  hard  antenna  problem  for  Hanscom  AFB.  Scott  Santarelli,  Tian-Li  Yu,  and  the 
PI  have  applied  hBOA  to  the  solution  of  a  parameter-tuning  problem  in  matching  a  Rotman  lens 
to  a  Butler  matrix.  hBOA  found  a  30db-down  solution  when  minimizing  the  maximum 
difference  between  main  and  side  lobes.  Even  after  a  million  function  evaluations,  a  simple  GA 
was  unable  to  find  an  acceptable  solution. 

9  orders  of  magnitude  speed  up  in  materials  modeling.  We  (Duane  Johnson,  Pascal  Bellon, 
Kumara  Sastry,  and  I)  have  applied  genetic  programming  to  multi-scale  modeling  of  alloys  using 
a  hybrid  of  the  molecular  dynamics  (MD)  and  kinetic  Monte  Carlo  (KMC)  procedures.  In  2D 
surface  modeling  our  calculations  project  speedups  of  9  orders  of  magnitude  at  300  degrees 
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Kelvin.  Two  papers  have  been  published  ( International  Journal  for  Multiscale  Computational 
Engineering  and  Physics  Review  B).  Simply  stated,  genetic  programming  is  used  to  perform 
customized  statistical  mechanics  by  bridging  the  different  time  scales  of  MD  and  KMC  quickly 
and  well. 

Speedups  in  multiscaling  for  chemistry,  too.  Work  involving  Todd  Martinez  (2005 
MacArthur  fellow),  Duane  Johnson,  and  the  PI  has  applied  multiobjective  GAs  and  model 
building  to  the  principled  speedup  of  excited  state  direct  dynamics  calculations  in  chemistry. 

This  work  is  providing  fresh  insights  and  important  research  avenues  toward  fast,  accurate 
chemistry  calculations  in  an  array  of  important  problem  areas. 

Multiobjective  hBOA  scalability  explored  in  hard  multiobjective  problems.  The  scalability 
of  multiobjective  estimation  of  distribution  algorithms  was  explored  with  fundamental  theory 
and  computational  experiments.  Depending  on  the  number  of  optima  that  need  to  be  tracked  in 
each  substructure,  it  is  easy  to  overwhelm  the  capacity  of  all  known  niching  algorithms  due  to 
simple  combinatorial  overload.  On  the  other  hand,  effective  niching  can  capably  preserve  a  large 
and  well  spaced  subset  of  the  optimal  set  or  it  can  preserve  the  full  Pareto  optimal  in  those  cases 
when  the  optimal  set  grows  in  a  manner  that  does  not  overwhelm  the  solver. 

Compact  classifier  system  invented.  The  idea  of  an  estimation  of  distribution  algorithm  has 
become  commonplace  in  optimization  problems,  but  these  ideas  have  not  been  widely  explored 
in  machine  learning  applications.  First  results  and  publications  were  prepared  on  the  compact 
classifier  system  that  uses  a  population  of  EDAs  to  capture  a  population  of  rules. 

BOA  used  to  speed  machine  learning.  Much  of  the  work  of  this  project  has  been  devoted  to 
optimization,  but  first  steps  were  made  to  use  competent  EDAs  to  learn  effective  substructure  in 
rule-learning  problem  domains.  In  difficult  hierarchical  machine  learning  problems,  the 
substructural  identification  procedures  proved  useful  in  solving  extraordinarily  hard 
classification  problems  quickly  and  well. 

From  bit  strings  to  real  codes  to  programs.  Using  mixtures  of  Gaussians  at  the  subspace 
level,  C.  W.  Ahn  has  built  an  effective  real  BOA  (rBOA)  that  tackles  problems  in  R"  quickly, 
reliably,  and  accurately.  Early  competent  GAs  solved  pseudo-boolean  problems  (problems  over 
bitstrings).  We  recently  carried  those  results  over  to  the  problem  of  scalable  genetic 
programming,  GAs  that  program  computers.  We  showed  that  competence  technique  reduces 
exponential  performance  to  subcubic  as  predicted  by  our  experience  with  bitstring  GAs. 
Experiments  are  underway  to  carry  these  results  over  to  practical  material  science  and  chemistry 
problems.  A  cornerstone  of  the  Illinois  approach  is  to  build  facetwise  models  of  critical 
phenomena.  A  key  to  our  success  in  GP  was  building  models  of  effective  statistical  decisions 
and  population  supply  for  GP. 

Organizationally  inspired  competent  GA  solves  hierarchically  difficult  problems. 

Techniques  using  dependency  structure  matrices  (DSMs)  have  been  used  by  large  corporations 
and  governmental  agencies  in  organizational  design  and  last  year  results  were  reported  on  using 
genetic  algorithms  for  DSM  clustering.  Those  results  led  previously  to  the  creation  of  a  GA 
called  the  DSMGA  using  DSM  clustering  to  do  building-block  decomposition.  A  US  patent 
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application  has  been  filed  for  both  the  GA  clustering  of  DSMs  and  the  use  of  DSMs  in  the 
creation  of  a  competent  GA.  This  year,  those  results  have  led  to  a  hierarchical  version  of  the 
DSMGA  that  solves  hierarchical  difficult  problems  through  explicit  substructure  compression. 

Little  quantitative  models  of  teams  (and  models).  The  style  of  modeling  used  to  advance 
competent  GA  design  has  led  to  the  creation  of  some  novel  quantitative  models  of  team  size, 
employment  frequency,  and  other  phenomena  using  simple  tradeoff  models  (decision  speed  vs. 
task  speed,  transaction  costs  versus  search  costs,  etc.).  These  little  models  parameterize 
important  quantities  in  organizations  that  are  usually  treated  by  qualitative  means  in  the  business 
literature.  A  paper  was  presented  (at  the  Academy  of  Management  Conference)  on  optimal 
department  size  in  an  organization  with  different  communication  rates  vertically  and 
horizontally.  A  recent  little  model  optimizes  the  tradeoff  in  modeling  between  accurate  models 
and  those  that  are  easily  used. 

Collaborative  systems  software  inspired  by  GAs.  Technical  innovation  in  GAs  has  led  to  a 
spinoff  project  in  collaborative  systems  design  inspired  by  competent  GA  ideas.  In  collaboration 
with  NCSA,  I  am  working  on  a  project  for  innovation  support  over  the  web  using  interactive 
GAs,  human-based  GAs,  and  chance  discovery.  The  project,  DISCUS  or  “Distributed  Innovation 
and  Scalable  Collaboration  in  Uncertain  Settings”  has  attracted  new  funding  and  a  surprising 
amount  of  attention.  A  US  patent  application  has  been  filed.  See  http://www-discus.qc.  uiuc.cdu/ 
for  additional  information.  Work  specifically  in  this  area  has  been  funded  under  the  recent 
AFOSR  IS&T  BAA. 

3  Personnel  Supported 

This  section  details  the  individuals  supported  on  this  project. 

Faculty  supported.  Professor  David  E.  Goldberg,  the  principal  investigator,  was  supported 
during  the  summers  of  2003-2005.  Duane  Johnson  (MatSE)  was  partially  supported  during  the 
summer  of  2004. 

Other  affiliated  vistors  postdoctoral  personnel.  The  following  is  a  list  of  visiting  faculty  or 
postdocs  affiliated  with  the  project.  Unsupported  affiliates  may  have  had  some  travel  or 
incidental  expenses  paid  by  the  project: 

1 .  Professor  Pier  Luca  Lanzi  (Politecnico  di  Milano,  Italy) 

2.  Dr.  Xavier  Llora  (Ramon  Llull  University,  Barcelona,  Spain) 

3.  Dr.  Hussein  Abbass  (University  of  New  South  Wales,  Australian  Defence  Force  Campus) 

4.  Dr.  Kei  Ohnishi  (Kyushu  University,  Japan) 

5.  Dr.  Naohiro  Matsumura  (Osaka  University,  Japan) 

Graduate  student  affiliates.  The  following  is  a  list  of  graduate  students  supported  or  affiliated 
with  the  project.  Unsupported  affiliates  may  have  had  some  travel  or  incidental  expenses  paid  by 
the  project: 

1.  Chen-Ju  Chao  (TRECC) 
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2.  Abhimanyu  Gupta  (NCASR) 

3.  Feipeng  Li  (TRECC) 

4.  Kazuhisa  Inaba,  (Tsukuba  University,  Japan) 

5.  Mohit  Jolly  (NCASR) 

6.  John  Loveall  (TRECC) 

7.  Kumara  Sastry  (MatSE) 

8.  Jai  Vasanth  (NCASR) 

9.  Paul  Winward  (AFOSR) 

10.  Tian-Li  Yu  (AFOSR) 

11.  Peng  Zang  (NCASR) 

Undergraduate  student  affiliates.  The  following  is  a  list  of  graduate  students  supported  or 
affiliated  with  the  project.  Unsupported  affiliates  may  have  had  some  travel  or  incidental 
expenses  paid  by  the  project: 

1 .  Karen  Czamecki  (AFOSR) 

2.  Jeffrey  Leesman  (TRECC) 

3.  Ryan  Spraetz  (AFOSR) 

4.  David  Ball  (AFOSR) 

4  Publications  for  2003 

Butz,  M.,  D.  E.  Goldberg,  and  K.  Tharakunnel.  Analysis  and  Improvement  of  Fitness 
Exploitation  in  XCS:  Bounding  Models,  Tournament  Selection,  and  Bilateral  Accuracy. 
Evolutionary  Computation,  1 1(3),  239-277  (2003). 

Cantu-Paz,  E.  and  D.  E.  Goldberg.  Are  Multiple  Runs  of  Genetic  Algorithms  Better  Than  One? 
GECCO-2003:  Proceedings  of  the  Genetic  and  Evolutionary  Computation  Conference,  801-812 
(2003). 

Chen,  Y.-P.,  and  D.  E.  Goldberg.  An  Analysis  of  a  Reordering  Operator  with  Tournament 
Selection  on  a  GA-hard  Problem.  GECCO-2003:  Proceedings  of  the  Genetic  and  Evolutionary 
Computation  Conference,  823-836  (2003). 

Chen,  Y.-P.  and  D.  E.  Goldberg.  Tightness  Time  for  the  Linkage  Learning  Genetic  Algorithm. 
GECCO-2003:  Proceedings  of  the  Genetic  and  Evolutionary  Computation  Conference,  837-849 
(2003). 

Espinoza,  F.,  B.  Minsker,  and  D.  E..  Performance  Evaluation  and  Population  Reduction  for  a 
Self-adaptive  Hybrid  Genetic  Algorithm  (SAHGA).  GECCO-2003:  Proceedings  of  the  Genetic 
and  Evolutionary  Computation  Conference,  922-933  (2003). 

Espinoza,  F.,  B.  S.  Minsker,  and  D.  E.  Goldberg.  Local  Search  Issues  for  the  Application  of  a 
Self-adaptive  Hybrid  Genetic  Algorithm  in  Groundwater  Remediation  Design.  ASCE 
Environmental  &  Waste  Resources  Institute  (2003). 
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Goldberg,  D.  E.  The  Design  of  Innovation:  Lessons  from  and for  Competent  Genetic  Algorithms. 
(Boston:  Kluwer  Academic  Publishers,  2002).  (Japanese  translation  in  preparation). 

Llora,  X.  and  D.  E.  Goldberg.  Bounding  the  Effect  of  Noise  in  Multiobjective  Learning 
Systems.  Evolutionary  Computation,  1 1(3),  279-298  (2003). 

Llora,  X.,  &  Goldberg,  D.  E.  Wise  Breeding  GA  via  Machine  Learning  Techniques  for  Function 
Optimization.  GECCO-2003:  Proceedings  of  the  Genetic  and  Evolutionary  Computation 
Conference,  1172-1183  (2003). 

Llora,  X.,  D.  E.  Goldberg,  I.  Traus,  I.,  amd  E.  Bernado.  Accuracy,  Parsimony,  and  Generality  in 
Evolutionary  Learning  Systems  via  Multiobjective  Selection.  Advances  in  Learning  Classifier 
Systems,  (2003). 

Pelikan,  M.  and  D.  E.  Goldberg.  A  Hierarchy  Machine:  Learning  to  Optimize  from  Nature  and 
Humans.  Complexity,  5(5),  36-45  (2003). 

Pelikan,  M.  and  D.  E.  Goldberg.  Hierarchical  BOA  Solves  Ising  Spin  Glasses  and  MAXSAT. 
GECCO-2003:  Proceedings  of  the  Genetic  and  Evolutionary  Computation  Conference,  1271- 
1282  (2003). 

Reed,  P.,  Minsker,  B.  S.,  &  Goldberg,  D.  E.  Simplifying  Mulitiobjective  Optimization:  An 
Automated  Design  Methodology  for  the  Nondominated  Sorted  Genetic  Algorithm-1 1.  Water 
Resources  Research.  39(1),  1 196  (2003). 

Rothlauf,  F.  and  D.  E.  Goldberg.  Redundant  Representations  in  Evolutionary  Computation. 
Evolutionary  Computation,  1 1(A),  381-415  (2003). 

Sastry,  K..,  and  D.  E.  Goldberg.  Probabilistic  Model  Building  and  Competent  Genetic 
Programming.  Genetic  Programming  Theory  and  Practice,  205-220  (2003). 

Sastry,  K.  and  D.  E.  Goldberg.  Scalability  of  Selectorecombinative  Genetic  Algorithms  for 
Problems  with  Tight  Linkage.  GECCO-2003:  Proceedings  of  the  Genetic  and  Evolutionary 
Computation  Conference  1332-1344  (2003). 

Sastry,  K.  U.-M.  O'Reilly,  D.  E.  Goldberg,  and  D.  J.  Hill.  Building-Block  Supply  in  Genetic 
Programming.  Genetic  Programming  Theory  and  Practice,  137-154  (2003). 

Singh,  A.,  B.  S.  Minsker,  and  D.  E.  Goldberg..  Combining  Reliability  and  Pareto  Optimality— An 
Approach  using  Stochastic  Multi-objective  Genetic  Algorithms.  ASCE  Environmental  &  Waste 
Resources  Institute  (2003) 

Tharakunnel,  K.,  M.  Butz,  and  D.  E.  Goldberg.  Towards  Building  Block  Propagation  in  XCS:  A 
Negative  Result  and  Its  Implications.  GECCO-2003:  Proceedings  of  the  Genetic  and 
Evolutionary  Computation  Conference,  1906-1917  (2003). 
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Yu,  T.-L.,  D.  E.  Goldberg,  and  A.  Yassine.  A  Genetic  Algorithm  Design  Inspired  by 
Organizational  Theory.  GECCO-2003:  Proceedings  of  the  Genetic  and  Evolutionary 
Computation  Conference,  1620-1621  (2003). 

Yu,  T.-L.,  D.  E.  Goldberg,  and  K.  Sastry.  Optimal  Sampling  and  Speed-up  for  Genetic 
Algorithms  on  the  Sample  OneMax  Problem.  GECCO-2003:  Proceedings  of  the  Genetic  and 
Evolutionary  Computation  Conference,  1554-1565  (2003). 

Yu,  T.-L.,  D.  E.  Goldberg,  A.  Yassine,  A.,  and  Y.-p  Chen.  Genetic  Algorithm  Design  Inspired  by 
Organizational  Theory:  Pilot  Study  of  a  Dependency  Structure  Matrix  Driven  Genetic 
Algorithm,  Proceedings  of  Artificial  Neural  Networks  in  Engineering  2003  (ANNIE  2003),  327- 
332  (2003). 

Yu,  T.-L.,  A.  Yassine,  and  D.  E.  Goldberg.  A  Genetic  Algorithm  for  Developing  Modular 
Product  Architectures.  Proceedings  of  the  ASME  2003  International  Design  Engineering 
Technical  Conferences,  15th  International  Conference  on  Design  Theory  and  Methodology 
(DETC2003),  DTM-48657  (2003). 

Yu,  T.-L.,  Y.-p.  Chen,  D.  E.  Goldberg,  and  J.-H.  Chen.  An  Adaptive  Sampling  Scheme  for 
Genetic  Algorithms  on  the  Sampled  OneMax  Problem.  Proceedings  of  Artificial  Neural 
Networks  in  Engineering  2003  (ANNIE  2003),  39-44  (2003). 

4  Publications  for  2004 

Ahn,  C.  W.  R.  S.  Ramakrishna,  and  D.  E.  Goldberg.  Real-coded  Bayesian  Optimization 
Algorithm.  GECCO-2004:  Proceedings  of  the  Genetic  and  Evolutionary  Computation 
Conference,  I,  840-851  (2004). 

Butz,  M.,  D.  E.Goldberg,  and  P.  L.  Lanzi.  Bounding  Learning  Time  in  XCS.  GECCO-2004: 
Proceedings  of  the  Genetic  and  Evolutionary  Computation  Conference,  2,  739-750  (2004). 

Butz,  M.,  D.  E.Goldberg,  and  P.  L.  Lanzi.  Gradient-based  Learning  Updates  Improve  SCS 
Performance  in  Multistep  Problems.  GECCO-2004:  Proceedings  of  the  Genetic  and 
Evolutionary  Computation  Conference,  2,  751-762  (2004). 

Chen,  Y.-p.  and  D.  E.  Goldberg.  Introducing  Subchromosome  Representations  to  the  Linkage 
Learning  Genetic  Algorithm.  GECCO-2004:  Proceedings  of  the  Genetic  and  Evolutionary 
Computation  Conference,  1,  971-982  (2004). 

Chen,  Y.-P.,  and  D.  E.  Goldberg.  Convergence  Time  for  the  Linkage  Learning  Genetic 
Algorithm.  Proceedings  of  the  IEEE  Conference  on  Evolutionary  Computation,  39-46  (2004). 

Goldberg,  D.  E.  The  Everyday  Engineering  of  Organizational  and  Engineering  Innovation. 
Adaptive  Computing  in  Design  and  Manufacture  VI,  1-10. 
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Llora,  X.,  K.  Ohnishi,  Y.-p.  Chen,  and  D.  E.  Goldberg.  Enhanced  Innovation:  A  Fusion  of 
Chance  Discovery  and  Evolutoinary  Computation  to  Foster  Creative  Processes  and  Decision 
Making.  GECCO-2004:  Proceedings  of  the  Genetic  and  Evolutionary  Computation  Conference, 
2,  1314-1315  (2004). 

Ohnishi,  K.,  K.  Sastry,  Y.-p.  Chen,  D.  E.  Goldberg.  Inducing  Sequentiality  using  Grammatical 
Genetic  Codes.  GECCO-2004:  Proceedings  of  the  Genetic  and  Evolutionary  Computation 
Conference,  1,  1427-1437(2004). 

Pedersen,  G.  and  D.  E.  Goldberg.  Dynamic  Uniform  Scaling  for  Multiobjective  Genetic 
Algortihms.  GECCO-2004:  Proceedings  of  the  Genetic  and  Evolutionary  Computation 
Conference,  2,  1 1-23  (2004). 

Santarelli,  S.,  D.  E.  Goldberg,  and  T.-L.  Yu.  Optimization  of  a  Constrained  Feed  Network  for  an 
Antenna  Array  using  Simple  and  Competent  Genetic  Algorithm  Techniques.  Workshop 
Proceedings  of  the  Genetic  and  Evolutionary  Computation  Conference  2004  (GECCO  2004), 
Military  and  Security  Application  of  Evolutionary  Computation  (MSAEC-2004)  (2004). 

Sastry,  K.,  Goldberg,  D.  E.  (2004).  Designing  Competent  Mutation  Operators  via  Probabilistic 
Model  Building  of  Neighborhoods.  GECCO-2004:  Proceedings  of  the  Genetic  and  Evolutionary 
Computation  Conference  2,  114-125  (2004). 

Sastry,  K.  and  D.  E.  Goldberg,  D.  E.  Let's  Get  Ready  to  Rumble:  Crossover  versus  Mutation 
Head  to  Head.  GECCO-2004:  Proceedings  of  the  Genetic  and  Evolutionary  Computation 
Conference,  2,  126-137  (2004). 

Sastry,  K.,  U.-M.  O'Reilly,  and  D.  E.  Goldberg.  Population  Sizing  for  Genetic  Programming 
Based  upon  Decision  Making.  Genetic  Programming  Theory  and  Practice,  137-154  (2004). 

Sastry,  K.,  M.  Pelikan,  and  D.  E.  Goldberg.  Efficiency  enhancement  of  genetic  algorithms  via 
building-block-wise  fitness  estimation.  Proceedings  of  the  IEEE  Conference  on  Evolutionary 
Computation,  720-727  (2004). 

Sastry,  K.,  M.  Pelikan,  and  D.  E.  Goldberg.  Efficiency  enhancement  of  probabilistic  model 
building  genetic  algorithms.  Workshop  Proceedings  of  GECCO-2004:  Optimization  by  Building 
and  Using  Probabilistic  Models. 

Yu,  T.-L.  and  D.  E.  Goldberg,  D.  E.  (2004).  Dependency  Structure  Matrix  Analysis:  Off-line 
Utility  of  the  Dependency  Structure  Matrix  Genetic  Algorithm.  GECCO-2004:  Proceedings  of 
the  Genetic  and  Evolutionary  Computation  Conference,  2,  355-366  (2004). 

Yu,  T.-L.  and  Goldberg,  D.  E.  (2004).  Toward  an  Understanding  of  the  Quality  and  Efficiency  of 
Model  Building  for  Genetic  Algorithms.  GECCO-2004:  Proceedings  of  the  Genetic  and 
Evolutionary  Computation  Conference,  2,  367-378  (2004). 
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5  Publications  2005 


Butz,  M.  V.,  K.  Sastry,  and  D.  E.  Goldberg.  Strong,  Stable,  and  Reliable  Fitness  Pressure  in 
XCS  due  to  Tournament  Selection.  Genetic  Programming  and  Evolvable  Machines,  6(1),  53- 
77  (2005). 

Butz,  M.  V.,  D.  E.  Goldberg,  and  P.-L.  Lanzi,  Gradient  Descent  Methods  in  Learning  Classifier 
Systems:  Improving  XCS  Performance  in  Multistep  Problems.  IEEE  Transaction  on 
Evolutionary  Computation,  9(5),  452-473  (2005). 

Butz,  M.V.,  M.  Pelikan,  X.  Llora,  and  D.  E.  Goldberg.  Extracted  Global  Structure  Makes  Local 
Building  Block  Processing  Effective  in  XCS.  GECCO-2005:  Proceedings  of  the  Genetic  and 
Evolutionary  Computation  Conference,  655-662  (2005).  Best  Paper  Nominee  EDA  Track. 

Chen,  J.-H.,  Ho,  S.-Y.,  and  D.  E.  Goldberg.  Quality-Time  Analysis  of  Multi-Objective 
Evolutionary  Algorithms.  GECCO-2005:  Proceedings  of  the  Genetic  and  Evolutionary 
Computation  Conference  1455-1462  (2005). 

Chen,  Y.-p.  and  D.  E.  Goldberg.  Convergence  Time  for  the  Linkage  Learning  Genetic 
Algorithm.  Evolutionary  Computation,  13(3),  279-302  (2005). 

Espinoza,  F.  P.,  B.  S.  Minsker,  and  D.  E.  Goldberg.  Adaptive  Hybrid  Genetic  Algorithm  for 
Groundwater  Remediation  Design.  Journal  of  Water  Resources  and  Planning  Management, 
131(1),  14-24(2005). 

Goldberg,  D.  E.  John  H.  Holland,  Facetwise  Models,  and  Economy  of  Thought.  In  L.  Booker, 
S.  Forrest,  M.  Mitchell,  &  R.  Riolo  (Eds.),  Perspectives  on  Adaptation  in  Natural  and  Artificial 
Systems  (pp.  57-69).  New  York:  Oxford  University  Press  (2005). 

Goldberg,  D.  E.  Foreword.  In  Abe,  A.,  and  Y.  Ohsawa  (Eds.),  Readings  in  Chance  Discovery 
(pp.  i-ii).  Adelaide,  Australia:  Advanced  Knowledge  International  (2005). 

Lanzi,  P.-L.,  D.  Loiacono,  S.  W. Wilson,  and  D.  E.  Goldberg.  XCS  with  Computable  Prediction 
for  the  Learning  of  Boolean  Functions.  Proceedings  of  the  2005  Congress  on  Evolutionary 
Computation  (CEC2005)  (2005). 

Lanzi,  P.-L.,  D.  Loiacono,  S.  W.  Wilson,  and  D.  E.  Goldberg.  XCS  with  Computable  Prediction 
in  Multistep  Environments.  GECCO-2005:  Proceedings  of  the  Genetic  and  Evolutionary 
Computation  Conference,  1859-1866  (2005). 

Lanzi,  P.-L.,  D.  Loiacono,  S.  W.  Wilson,  and  D.  E.  Goldberg.  Extending  XCSF  Beyond  Linear 
Approximation.  GECCO-2005:  Proceedings  of  the  Genetic  and  Evolutionary  Computation 
Conference  1827-1834  (2005). 

Lima,  C.  F.,  K.  Sastry,  D.  E.  Goldberg,  and  F.  G.  Lobo.  Combining  Competent  Crossover  and 
Mutation  Operators:  A  Probabilistic  Model  Building  Approach.  GECCO-2005:  Proceedings  of 
the  Genetic  and  Evolutionary  Computation  Conference,  735-742  (2005). 

Llora,  X.,  K.  Sastry,  D.  E.  Goldberg,  Binary  Rule  Encoding  Schemes:  A  Study  Using  the 
Compact  Classifier  System.  Workshop  Proceedings  of  the  Genetic  and  Evolutionary 
Computation  Conference,  88-89  (2005). 
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Llora,  X.,  K.  Sastry,  D.  E.  Goldberg,  The  Compact  Classifier  System:  Motivation,  Analysis, 
and  First  Results  [poster].  GECCO-2005:  Proceedings  of  the  Genetic  and  Evolutionary 
Computation  Conference  [poster],  1993-1994  (2005). 

Llora,  X.,  K.  Sastry,  D.  E.  Goldberg,  The  Compact  Classifier  System:  Motivation,  Analysis, 
and  First  Results.  Proceedings  of  the  2005  Congress  on  Evolutionary  Computation  (CEC2005) 
(2005). 

Llora,  X.,  K.  Sastry,  D.  E.  Goldberg,  A.  Gupta,  L.  Lakshmi.  Combating  User  Fatigue  in  iGAs: 
Partial  Ordering,  Support  Vector  Machines,  and  Synthetic  Fitness.  GECCO-2005:  Proceedings 
of  the  Genetic  and  Evolutionary  Computation  Conference,  1363-1370  (2005). 

Matsumura,  N.,  D.  E.  Goldberg,  D.E.,  and  X.  Llora,  Mining  Social  Networks  in  Message 
Boards.  Proceedings  of  the  A1SB  Symposium  on  Conversational  Informatics  for  Supporting 
Social  Intelligence  &  Interaction  (2005).  hup://kaiwa.ristex. ist.go.jp/AISB  Cl/ 

McClay,  L  and  D.  E.  Goldberg.  Efficient  Genetic  Algorithms  using  Discretization  Scheduling. 
Evolutionary  Computation,  13(3),  353-385  (2005). 

Pelikan,  M.,  K.  Sastry,  and  D.  E.  Goldberg,  Multiobjective  hBOA.  Clustering,  and  Scalability. 
GECCO-2005:  Proceedings  of  the  Genetic  and  Evolutionary  Computation  Conference,  663- 
670  (2005). 

Qurin,  A.,  J.  Korczak,  M.  V.  Butz,  and  D.  E.  Goldberg.  Analysis  and  Evaluation  of  Learning 
Classifier  Systems  Applied  to  Hyperspectral  Image  Classification.  Proceedings  of  the  Fifth 
International  Conference  on  Intelligent  Systems  Design  and  Applications  (2005). 

Sastry,  K.,  H.  A.  Abbass,  D.  E.  Goldberg,  and  D.  D.  Johnson.  Sub-structural  Niching  in 
Estimation  of  Distribution  Algorithms.  GECCO-2005:  Proceedings  of  the  Genetic  and 
Evolutionary  Computation  Conference,  671-678  (2005).  Best  Paper  Nominee,  EDA  Track. 

Sastry,  K.,  D.  D.  Johnson,  D.  E.  Goldberg,  and  P.  Bellon.  Genetic  Programming  for  Multi- 
Timescale  Modeling:  A  Proof-of-Principle  Study.  Physics  Review  B,  72(8),  085438,  9  (2005). 

Sastry,  K..,  M.  Pelikan,  and  D.  E.  Goldberg,  Decomposable  Problems,  Niching,  and  Scalability 
of  Multiobjective  Estimation  of  Distribution  Algorithms.  Proceedings  of  the  2005  Congress  on 
Evolutionary  Computation  (CEC2005)  (2005). 

Yassine,  A.,  D.  E.  Goldberg,  and  T.-L.  Yu,  Simple  Models  of  Hierarchical  Organizations. 
Proceedings  of  the  Academy  of  Management  (2005). 

Yu,  T.-L.,  K.  Sastry,  and  D.  E.  Goldberg,  Linkage  Learning,  Overlapping  Building  Blocks,  and 
a  Systematic  Strategy  for  Scalable  Recombination.  GECCO-2005:  Proceedings  of  the  Genetic 
and  Evolutionary  Computation  Conference,  1217-1224  (2005). 

Yu,  T.-L.,  K.  Sastry,  and  D.  E.  Goldberg,  Online  Population  Size  Adjusting  Using  Noise  and 
Substructural  Measurements.  Proceedings  of  the  2005  Congress  on  Evolutionary  Computation 
(CEC2005)  (2005). 
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6  Publications  2006 


Alias,  F.  X.  Llora,  L.  Formiga,  K.  Sastry,  and  D.  E.  Goldberg,  Evaluation  Consistency  in  iGAs: 
User  Contradictions  as  Cycles  in  partial-Ordering  Graphs.  Proceedings  of  the  IEEE 
International  Conference  on  Acoustics,  Speech,  and  Signal  Processing  (ICASSP  2006),  865- 
868  (2006). 

Butz,  M.V.,  M.  Pelikan,  X.  Llora,  and  D.  E.  Goldberg,  Automated  Global  Structure  Extraction 
For  Effective  Local  Building  Block  Processing  in  XCS.  Evolutionary  Computation,  14(3),  345- 
380  (2006). 

Lanzi  P.L.,  D.  Loiacono,  S.  W.  Wilson,  and  D.  E.  Goldberg,  Classifier  Prediction  based  on  Tile 
Coding.  GECCO-2006:  Proceedings  of  the  Genetic  and  Evolutionary  Computation 
Conference,  1497-1504  (2006).  Best  paper  award  LCS/GBML  track. 

Lanzi  P.L.,  D.  Loiacono,  S.  W.  Wilson,  and  D.  E.  Goldberg,  Prediction  Update  Algorithms  for 
XCSF:  RLS,  Kalman  Filter,  and  Gain  Adaptation.  GECCO-2006:  Proceedings  of  the  Genetic 
and  Evolutionary  Computation  Conference,  1502-1512  (2006). 

Llora,  X.,  D.  E.  Goldberg,  Y.  Ohsawa,  N.  Matsumura,  Y.  Washida,  H.  Tamura,  M.  Yoshikawa, 
M.  Welge,  L.  Auvil,  D.  Searsmith,  K.  Ohnishi,  and  C.-J.  Chao,  Innovation  and  creativity 
support  via  chance  discovery,  genetic  algorithms,  and  data  mining.  New  Mathematics  and 
Natural  Computation,  2(1),  85-100  (2006). 

Llora,  X.,  K.  Sastry,  F.  Alias,  D.  E.  Goldberg,  and  M.  Welge,  Analyzing  Active  Interactive 
Genetic  Algorithms  using  Visual  Analytics  [poster],  GECCO-2006:  Proceedings  of  the 
Genetic  and  Evolutionary  Computation  Conference,  1417-1418  (2006). 

Matsumura,  N.,  D.  E.  Goldberg,  and  X.  Llora,  Communication  Gap  Management  for  Fertile 
Community.  Journal  of  Soft  Computing  (in  press). 

Pelikan,  M.,  K.  Sastry,  and  D.  E.  Goldberg,  Sporadic  Model  Building  for  Efficiency 
Enhancement  of  hBOA.  GECCO-2006:  Proceedings  of  the  Genetic  and  Evolutionary 
Computation  Conference,  405-412  (2006). 

Santarelli  S.,  T.-L.  Yu,  D.  E.  Goldberg,  E.  Altshuler  E.,  T.  O’Donnell,  H.  Southall,  and  R. 
Mailloux,  Military  Antenna  Design  Using  Simple  and  Competent  Genetic  Algorithms. 
Mathematical  and  Computer  Modelling,  43(9-10),  990-1022  (2006). 

Sastry,  K.,  P.  Winward,  D.  E.  Goldberg,  and  C.  Lima,  Fluctuating  Crosstalk  as  a  Source  of 
Deterministic  Noise  and  Its  Effects  on  GA  Scalability.  Proceedings  of  the  2006  EvoWorkshops, 
740-751  (2006). 

Sastry,  K.,  D.  D.  Johnson,  A.  L.  Thompson,  D.  E.  Goldberg,  T.  J.  Martinez,  J.  Leiding,  and  J. 
Owens,  Multiobjective  Genetic  Algorithms  for  Multiscaling  Excited  State  Direct  Dynamics  in 
Photochemistry.  GECCO-2006:  Proceedings  of  the  Genetic  and  Evolutionary  Computation 
Conference,  1745-1752  (2006).  Best  paper  real-world  applications  track. 

Sastry,  K.,  C.  F.  Lima,  and  D.  E.  Goldberg,  Evaluation  Relaxation  Using  Substructural 
Information  and  Linear  Estimation.  GECCO-2006:  Proceedings  of  the  Genetic  and 
Evolutionary  Computation  Conference,  419-426  (2006). 
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Tsutsui,  S.,  M.  Pelikan,  and  D.  E.  Goldberg  .  Probabilistic  Model-building  Genetic  Algorithms 
using  Histogram  Models  in  Continuous  Domain.  Journal  of  the  Information  Processing 
Society  of  Japan  (in  press). 

Winward,  P.  and  D.  E.  Goldberg,  Fluctuating  Crosstalk,  Deterministic  Noise,  and  GA 
Scalability,  GECCO-2006:  Proceedings  of  the  Genetic  and  Evolutionary  Computation 
Conference,  1361-1368(2006). 

Yu,  T.-L.  and  D.  E.  Goldberg,  Conquering  Hierarchical  Difficulty  by  Explicit  Chunking: 
Substructural  Chromosome  Compression,  GECCO-2006:  Proceedings  of  the  Genetic  and 
Evolutionary  Computation  Conference ,  1385-1392  (2006). 


6  Interactions  and  Transitions 

This  section  lists  meeting  participation,  presentations,  and  transitions. 

Meetings  &  Presentations 

All  conference  papers  above  represent  presentations  by  Professor  Goldberg,  his  affliates,  or  his 
students.  Additionally,  Professor  Goldberg  gave  many  keynote  talks  and  tutorials  during  the 
grant  period. 

-  EuroGP  2003,  Colchester,  UK,  2003  (keynote) 

-  Introductory  Tutorials  in  Optimization  and  Search  Methodologies,  Nottingham,  UK, 
2003  (tutorial) 

-  7th  Joint  Conference  on  Information  Sciences,  Research  Triangle,  NC,  2003  (keynote) 

-  University  of  Vermont  Workshop  on  Evolutionary  Computation,  2003  (keynote) 

-  Spencer  &  Spencer  Systems  Mathematics  and  Computer  Science  Lecture,  UMSL,  2003 
(invited  lecture) 

-  GECCO-2003,  Chicago,  IL,  2003  (tutorial) 

-  ACDM  Lecture,  Adaptive  Computing  in  Design  and  Manufacture,  Bristol,  UK,  2004 
(keynote) 

-  CIRP  Keynote,  CIRP  Design  Seminar,  Shanghai,  China,  2005  (keynote). 

-  ICIP  Keynote,  IEEE  International  Conference  on  Image  Processing,  Genoa,  Italy,  2005 
(keynote). 

-  Joint  Australian  Conference  on  Artificial  Intelligence  &  Australian  Conference  on 
Artificial  Life,  Keynote,  Sydney,  2005. 

-  Frontiers  of  Computational  Science,  Keynote,  2005. 

Transitions 

Competent  GA  techniques  transferred  to  Hanscom  AFB.  As  part  of  this  project,  hBOA  and 
other  competent  GA  techniques  were  used  in  concert  with  Hanscom  AFB  personnel  to  improve 
the  performance  of  evolutionary  antenna  design.  hBOA  proved  pivotal  in  obtaining  results  in  a 
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difficult  problem  where  simple  GAs  did  not  succeed.  A  1  Vi  short  course  was  held  in  Urbana,  IL 
for  Scott  Santarelli  and  Terry  O’Donnell  from  Hanscom  earlier  this  summer. 

hBOA  licenses  in  the  works.  The  hierarchical  Bayesian  optimization  algorithm  is  being  under 
evaluation  license  for  applications  in  stock  market  decision  making.  Test  results  are  promising 
and  a  decision  on  a  full  license  of  the  technology  is  imminent.  A  license  of  hBOA  to  a  firm  in 
electronic  design  automation  is  in  the  final  stages  of  negotiation. 

Nextumi  one-year  old.  With  venture  capital  provided  by  IllinoisVentures  and  Blue  Chip 
Venture  Capital,  a  ubiquitous  recommendation,  introduction,  and  personalization  company  called 
Nextumi  was  formed  one  year  ago.  Nextumi  licensed  AFOSR  sponsored  competent  GA 
research  (patents)  to  help  to  create  consumer-adaptive  technology. 

Kluvver  Book  Series  Merges.  Due  to  the  merger  of  Springer  and  Kluwer,  the  Pi’s  series  in 
Genetic  and  Evolutionary  Computation  will  now  be  published  as  part  of  the  Springer  nameplate 
hltp:/Avww.sprinccronlinc.eom/saw/cda/fronlpagc/0.  1  1 855,4-401 09-69-33 1  1021 3-0,00.html  and 

the  series  will  be  merged  with  John  Koza’s  series  on  Genetic  Programming.  A  revamping  of  the 
series  in  line  with  Springer  practices  in  the  works. 


7  New  Discoveries,  Inventions,  or  Patent  Disclosures 

Discoveries  &  inventions 

Many  of  the  papers  above  represent  new  inventions  and  discoveries. 

Patents 

Five  patents  have  been  filed  in  connection  with  AF  research  in  the  recent  past: 

Llora,  X.,  Sastry,  K.  and  D.  E.  Goldberg,  Methods  for  interactive  computing,  US  utility 
patent  pending. 

Pelikan,  M.  and  D.  E.  Goldberg.  Method  for  optimizing  a  solution  set.  US  utility  patent 
7047169  granted  (2006). 

Pelikan,  M,  Sastry,  K.,  and  D.  E.  Goldberg.  Methods  for  efficient  solution  set 
optimization.  US  utility  patent  pending. 

Goldberg,  D.  E.,  T.-L.  Yu,  and  A.  Yassine.  Methods  and  program  products  for 
optimizing  problem  clustering.  US  utility  patent  pending. 

Goldberg,  D.  E.,  M.  Welge,  &  X.  Llora,  X.  Methods  and  systems  for  collaboration, 
decision  support,  and  knowledge  management.  US  utility  patent  pending. 
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Two  additional  disclosures  are  in  preparation. 


6  Interactions  and  Transitions 

This  section  lists  meeting  participation,  presentations,  and  transitions. 

Meeting  Participation  and  Presentation 

All  conference  papers  above  represent  presentations  by  Professor  Goldberg,  his  affliates,  or  his 
students.  Additionally,  Professor  Goldberg  gave  numerous  keynote  talks  and  tutorials  during 
2003-2004: 

Transitions 

Competent  GA  techniques  transferred  to  Hanscom  AFB.  As  part  of  this  project,  hBOA  and 
other  competent  GA  techniques  were  used  in  concert  with  Hanscom  AFB  personnel  to  improve 
the  performance  of  evolutionary  antenna  design.  hBOA  proved  pivotal  in  obtaining  results  in  a 
difficult  problem  where  simple  GAs  did  not  succeed. 

2  orders  of  magnitude  faster  and  an  order  of  magnitude  better.  Competent  GA  techniques 
were  used  by  the  PI  and  two  of  his  students  in  an  extended  consulting  engagement  for  an  Israeli 
company  to  improve  optimization  of  cellular  phone  networks. 

ISGEC  Evaluating  SIGEVO.  The  PI  was  founding  chair  (1999)  of  the  International  Society 
for  Genetic  and  Evolutionary  Computation.  That  society  has  been  invited  by  the  Assocation  for 
Computing  Machinery  (ACM)  to  become  a  SIG  (special  interest  group)  as  part  of  ACM.  A 
decision  whether  to  join  or  not  should  be  reached  by  fall. 

Kluwer  Book  Series.  Due  to  the  merger  of  Springer  and  Kluwer,  the  Pi’s  series  in  Genetic  and 
Evolutionary  Computation  will  now  be  published  as  part  of  the  Springer  nameplate 

hiin:/Av\\\\  .spriiu’cronlinc.com/sRw/cda/frontixmc/0, 1  1 855.4-401 09-69-33 1  102 1 3-0.00.html. 

Innovation  course  on  line.  The  Pi’s  book,  The  Design  of  Innovation,  has  been  turned  into  an 
eight-lecture  online  shortcourse  (http://onlinc.cnur.uiuc.edu/shortcourses/innovation) 

Nextumi  started.  With  venture  capital  provided  by  IllinoisVentures  and  Blue  Chip  Venture 
Capital,  a  social  networking  company  called  Nextumi  has  been  formed.  Competent  GA  research 
will  be  used  to  create  consumer-adaptive  technology. 


8  Honors  and  Awards 

JSD  Professor.  Professor  Goldberg  was  named  Jerry  S.  Dobrovolny  Distinguished  Professor  in 
Entrepreneurial  Engineering  in  May  2003.  The  investiture  was  held  September  23.  2003. 
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ISGEC  Fellow.  Professor  Goldberg  was  named  part  of  the  inaugural  class  of  Senior  Fellows  for 
the  International  Soceiety  for  Genetic  and  Evolutionary  Computation. 

Other  Awards.  1985  NSF  Presidential  Young  Investigator.  1995,  Associate,  Center  for 
Advanced  Study  (Illinois).  1996,  Wickenden  Award  (ASEE).  1997,  Gambrinus  Fellow 
(Dortmund). 

Student  &  Affiliate  Honors 

2003  CSE  fellowship  to  Butz.  Martin  Butz  won  a  2003-2004  Computational  Science  and 
Engineering  fellowship  at  the  UIUC  for  his  work  in  anticipation  and  genetics-based  machine 
learning. 

GECCO-2003  best  paper.  Martin  Butz  and  Kumara  Sastry  won  best  paper  in  classifier  systems 
(LCS)  at  the  GECCO-02003  for  their  paper  “Tournament  Selection:  Stable  Fitness  Pressure  in 
XCS”  (with  D.  E.  Goldberg). 

GECCO-2004  best  paper  nominations.  Three  papers  from  lab  members  were  nominated  for 
best  paper  at  the  2004  Genetic  and  Evolutionary  Computation  conference. 

GECCO-2005  best  paper  nominations.  Two  papers  from  lab  members  were  nominated  for 
best  paper  at  the  2004  Genetic  and  Evolutionary  Computation  conference. 

GECCO-2005  best  papers  and  nominations.  Two  lab  papers  won  best  paper  awards  in  2006. 

Silver  Hurnie  Award.  AFOSR  sponsored  work  won  a  Silver  Award  in  the  Human  Competitive 
Results  competition  at  2006  GECCO  conference.  Team  members,  Kumara  Sastry,  Duane  D. 
Johnson,  Alexis  L.  Thompson,  David  E.  Goldberg,  Todd  J.  Martinez,  Jeff  Leiding,  and  Jane 
Owens  received  a  $3000  prize  for  their  work  on  Multiobiective  Genetic  Algorithms  for 
Mulliscaline  F.xcited-Slatc  Direct  Dynamics  in  Photochemistry. 


14 


